Stochastic Dynamics of Invasion and Fixation 
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We study evolutionary game dynamics in finite populations. We analyze an evolutionary process, which 
we call pairwise comparison, for which we adopt the ubiquitous Fermi distribution function from statistical 
mechanics. The inverse temperature in this process controls the intensity of selection, leading to a unified 
framework for evolutionary dynamics at all intensities of selection, from random drift to imitation dynamics. 
We derive, for the first time, a simple closed formula which determines the feasibility of cooperation in finite 
populations, whenever cooperation is modeled in terms of any symmetric two-person game. In contrast with 
previous results, the present formula is valid at all intensities of selection and for any initial condition. We 
investigate the evolutionary dynamics of cooperators in finite populations, and study the interplay between 
intensity of selection and the remnants of interior fixed points in infinite populations, as a function of a given 
initial number of cooperators, showing how this interplay strongly affects the approach to fixation of a given 
trait in finite populations, leading to counter-intuitive results at different intensities of selection. 
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More than thirty years have passed since John Maynard- 
Smith laid the foundations of Evolutionary Game Theory 
In its context, the replicator dynamics equation Q 01 has 
inspired mathematicians, physicists, biologists, economists, 
psychologists, as well as political and computer scientists 
to investigate the evolution of traits under natural selection 
Hill. In the absence of mutations, and whenever pop- 
ulations are infinite, homogeneous and well-mixed (each in- 
dividual is equally likely to interact with any other individ- 
ual), the (mean-field) description by the deterministic replica- 
tor dynamics in which the abundance a; of a strategy changes 
according to its relative payoff, x = x(ir x — (w)), has led to 
inumerous insights. 

In spite of the insights provided by deterministic replicator 
dynamics, it has been long recognized that populations, being 
finite, will exhibit stochastic effects which may play a siz- 
able role. Such behaviour is well-known in physics, where 
stochastic effects in finite sized systems have been studied 
for a long time 0]. Studies of stochastic evolutionary 
dynamics in finite populations, however, have been carried 
out mostly numerically. Indeed, finite-size populations are 
an ever-present ingredient in individual-based computer sim- 
ulations (I3[Tl[ll[ll[ll[Tl[ll[Tl[ll, which naturally 
incorporate such stochastic effects. Different intensities of se- 
lection have been employed in these studies, ranging from a 
strong selection framework, captured by the finite-population 
analogue of replicator dynamics foll4l[l5lllrlll7|l to an ex- 
treme selection pressure under imitation dynamics, used as a 
metaphor of cultural evolution |4|. 

Recently, the fixation probability of a trait under frequency 
dependent selection in a finite population has been derived 
IU9I1 . In order to control the interplay of drift and selection, 
an intensity of selection has been proposed, which measures 
the impact of the game on the fitness and determines the am- 
plitude of fitness differences on which selection acts. In such 
a system, fitness is not the number of offspring, but a mea- 



sure for the potential to reproduce. Analytical results were 
obtained for weak selection, when the result of the game acts 
as a linear perturbation to neutral drift. The biological rel- 
evance of this weak selection stems from the fact that most 
evolutionary changes are near neutral, i.e. have small influ- 
ence on fitness 12(1. Therefore, neutral selection can serve as 
a reference point Il9l l2lll . It has been shown that the finite- 
ness of populations may indeed lead to fundamental changes 
in the conventional picture emerging from deterministic repli- 
cator dynamics in infinite populations. For instance, a single 
cooperator in a Prisoner's Dilemma has now a small, yet non- 
zero probability of survival in a finite population of defectors. 
Conversely, it is not always certain that defectors wipe out co- 
operators in finite populations ll22ll23ll . 

In physics, stochastic effects are often described in terms 
of an effective temperature, a methodology which has proven 
both successful and insightful at all scales. In line with this 
long tradition, we introduce here a temperature of selection in 
the evolutionary game dynamics of finite populations. Such 
temperature controls the balance between selection and ran- 
dom drift in finite populations, providing a convenient frame- 
work to study evolutionary game dynamics at all intensities of 
selection - from neutral, random drift, up to the extreme limit 
of cultural imitation dynamics. We define the pairwise com- 
parison process, which utilizes the ubiquitous Fermi distribu- 
tion function at finite temperature. Making use of the Fermi 
distribution, we provide for the first time a closed analytical 
expression for the fixation probability in a finite, well-mixed 
population of arbitrary size, for an arbitrary number of co- 
operators at start, and for an arbitrary intensity of (frequency 
dependent) selection. As a result, we are able to explore the 
role of the temperature of selection in evolutionary game dy- 
namics in what concerns the overall fixation probabilities for 
different dilemmas. 

General symmetric two player games can be interpreted in 
terms of social dilemmas of cooperation I24l l25ll26ll27ll . in 
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which cooperators and defectors interact via the payoff matrix 
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In all dilemmas mutual cooperation is favored (R > P), al- 
though individual reasoning often leads to mutual defection. 
In the Prisoner's Dilemma (T > R > P > S), the only stable 
evolutionary equilibrium of the replicator dynamics is the sit- 
uation in which all individuals are defectors. Other dilemmas 
of cooperation are more favorable to cooperation, though. For 
instance, whenever T > R > S > P, we enter the realm 
of the Snowdrift Game, where it is best to do the opposite of 
what the other player does: to cooperate if the other player 
defects and to defect if the other player cooperates [ 33|. This 
leads to the stable coexistence of cooperators and defectors 
under replicator dynamics, at the cooperator frequency 
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In other words, starting from any given non-zero frequency 
of cooperators in the population, the system will evolve to- 
wards this coexistence equilibrium state. A third dilemma 
arises whenever R > T > P > S, the so-called Stag-Hunt 
game. In this case, one is better off doing whatever the other 
player will do, which corresponds to a coordination game fa- 
voring mutual cooperation (R). Indeed, replicator dynamics 
predicts the existence of an unstable interior fixed point at x* 
given by Eq. (0, such that for frequencies above this point the 
population will evolve towards 100% cooperation, the other 
fate being associated with startup frequencies below the un- 
stable fixed point. 

As usual in Evolutionary Game Theory, the fitness of each 
individual is associated with the payoff resulting from interac- 
tions with other individuals from the population. Under strong 
selection, an individual with higher fitness will always replace 
an individual with lower fitness. Let us consider the follow- 
ing process : Two individuals are chosen randomly from the 
entire population - for example a cooperator C and a defec- 
tor D, with payoffs ttc and 7Td, respectively. Under imita- 
tion dynamics, the fitter replaces the less fit with probability 
p = 1. Similarly, under replicator dynamics selection always 
increases the fraction of the fittest type. However, most evolu- 
tionary changes have a small impact on fitness and sometimes 
the less successful individual may actually replace the more 
successful one. Whereas such occasional errors may be safely 
neglected in large populations, they can have decisive effects 
in finite populations |19]. These effects are nicely captured 
by replacing the probability p above with the Fermi function 
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which gives the probability that C replaces D. Such a pro- 
cess we call Pairwise Comparison. The inverse temperature 
(3 > controls here the intensity of selection. For (3 — > oo, 



the situation described above is recovered, i.e. the probability 
for replacement of the less fit individual is one. However, for 
any finite (3, the reverse process may actually take place with 
a non-vanishing probability given by 1 — p. In particular, it is 
interesting to consider this process for (3 <C 1, in which the fit- 
ness has a marginal influence on replacement. For j3 -C 1, the 
process described above reduces exactly to the frequency de- 
pendent Moran process under weak selection, studied in 1 19]. 

Let us consider the general payoff matrix Eq. (??). When 
the number of interactions per individual is very high, the pay- 
offs will depend only on the fraction of both types in the pop- 
ulation. If there are j cooperators and N ~ j defectors, they 
will accumulate payoffs ttc = (j — 1)R + (N — j)S and 
tt d = jT + (N - j - 1)P, respectively Self interac- 
tions are excluded. Under Pairwise Comparison, and in the 
absence of mutations, only when the two individuals chosen 
have different strategies the total number of individuals with 
a given strategy can change by one. Similarly to the Moran 
process described in 1 19], this defines a finite state Markov 
process with an associated tri-diagonal transition matrix. The 
probabilities to increase the number of cooperators from j to 
j + 1, Tj, and the probability to decrease that number from j 

to j - 1, T7 are 
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For large populations, this process can be approximated by a 
stochastic differential equation with drift T. — T7 and dif- 

T7)/N l30ll . For the pairwise comparison 
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process, this yields 
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where x — i /N is the fraction of cooperators and £ is Gaus- 
sian white noise with variance one. For N —> oo, the stochas- 
tic term vanishes. In this limit, the replicator dynamics |4] 
is recovered from the first order term of a high temperature 
expansion, (3 <C 1. 

In finite populations, the quantity of interest in this process 
is the fixation probability of cooperators, i.e. the probability 
to end up in the state with N cooperators given that the ini- 
tial number is k. This probability depends only on the ratio 
o,j = Tr /Tj~ '. For the Pairwise Comparison process, this ra- 
tio reduces to ctj — exp [— (3{irc — 7Tp )1- In terms of a,-, the 
fixation probability is given by 
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and the payoff difference can be written as ire — ttd = 2uj + 
2v, where 2u = R-S-T+P and2« = -R+SN-PN+P, 
which simplifies the fixation probability to 
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One can interpret the sums in the previous equation as nu- 

k— - 

merical approximations to the integral f i 2 Yl^i a jdi- The 

error of this approximation is of order TV~ 2 and will therefore 
rapidly vanish with increasing TV. Replacing the sums by the 
integrals leads to the surprisingly simple formula, 
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is the er- 
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For u — 0, which is 
in Game Theory sometimes referred to as "equal gains from 
shifting", we find instead 
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This result is formally identical to the fixation probability of 



k individuals with fixed relative fitness 



Eqs. and (jSJl reduce trivially to the neutral selection re- 
sult = k/N for f3 — > 0. In contrast with the results from 
Ref. 1 19], these closed expressions are valid for any intensity 
of selection and for any given initial number of individuals of 
a given trait. Two limiting cases follow immediately: (3 <C 1 
corresponds to weak selection and yields the generalization 
of the weak selection result for the birth-death process given 
in 11911 to an arbitrary initial number of mutants. Strong se- 
lection is described by (3 3> 1 and reduces the process to a 
semi-deterministic imitation process: The speed of this pro- 
cess remains stochastic, but the direction always increases in- 
dividual fitness for j3 — > oo. This limit, which corresponds to 
the cultural evolution models of imitation dynamics, is outside 
the realm of the birth-death process studied in 1 19] and results 
from the nonlinearity of the Fermi distribution function. 

In the following we study the evolutionary game dynam- 
ics of finite populations in some concrete cases, in connection 
with the results shown in Fig. 1. 

For the Prisoner's Dilemma we choose R = 3, S = 1, 
T = 4, and P = 2. Since u = 0, the fixation probability is 
given by Eq. In Fig. la we show a comparison between 
the analytical results and extensive computer simulations for a 
small population of TV = 20 individuals. Clearly, cooperators 
are always in disadvantage with respect to defectors. In spite 
of the exponential increase of the fixation probability of coop- 
erators with the initial number k, only for very weak selection 
(/3 = 0.01) do cooperators acquire reasonable chances in a 
population as small as TV = 20. As stated before, the error 
of the approximation leading to Eqs. |7} and is of order 
N~ 2 . However, even for TV = 20 excellent agreement with 
numerical simulations is obtained as illustrated in Fig. 1. 

As a second numerical example, we consider the Stag-Hunt 
game given by the ranking of the payoff matrix illustrated 
in Fig. lb (note that we only changed the order of numer- 
ical values from Fig. la). This system exhibits two pure 
Nash equilibria, i.e. each strategy is the best reply to itself, 
and any selection mechanism leads to a bistable outcome. 
The replicator dynamics predicts a sharp transition from full- 
defection to full-cooperation at the unstable fixed point x* 
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FIG. 1: Fixation probabilities for the three qualitatively different 
dilemmas of cooperation describable by 2 x 2 games. Simulation 
results (filled symbols) coincide perfectly with the theoretical result 
Eqs. Q an d {5} (solid lines). For comparison, the theoretical result 
for the Moran process at maximum selection intensity (w — 1, open 
circles) and the neutral selection result (f3 — 0, diagonal line) are 
shown, (a) In the Prisoner's Dilemma, defection is dominant. As 
a result, only for very weak selection (/? = 0.01) cooperators ac- 
quire significant chances of fixation, (b) The Stag-Hunt game has an 
unstable equilibrium at k* = (iV + 2) /2, and the transition across 
this point depends sensitively on the intensity of selection, (c) The 
Snowdrift Game exhibits an internal equilibrium at k* = (iV — 2) /2, 
which is an inflection point in the theoretical curves. A small popula- 
tion size of TV = 20 has been chosen on purpose to stress the quality 
of the theoretical result related to Eqs. and ^8). Each simulation 
result corresponds to the fraction of fixation of cooperators in 10 4 
independent realizations. 



given by Eq. 0. In finite populations, this point is given by 
k* = Nx* + (P - R)/(T + S - R- P), where the second 
term resulting from the exclusion of self interactions is irrele- 
vant for large TV. The transition at k* depends sensitively on 
the intensity of selection, becoming sharper the larger the in- 
tensity of selection. For the Moran process discussed in fl^l 
a smooth transition from zero to one is always observed in all 
dilemmas, even when the intensity of selection is maximized 
{w = 1), for which the fitness equals the payoff resulting from 
the game (open-circles in Fig. 1). Hence, only for very smooth 
transitions between the two absorbing states do the predictions 
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in fl9ll apply. Eq. Q, instead, provides the overall scenario 
for all intensities of selection without need to invoke the so- 
called 1/3-rule. 

The Snowdrift Game 13311 (also known as 'Hawk-Dove' 
game or 'Chicken') characterized by the payoff ranking T > 
R > S > P presents the most interesting scenario in this 
context. This system exhibits a mixed Nash equilibrium at 
k*. Replicator dynamics predicts that any initial condition 
will lead to this mixed-equilibrium. This is clearly not possi- 
ble in a finite population, in which the system ends up in one 
of the absorbing states k = or k = N. For weak selection, 
the fixation probability increases smoothly with k from zero to 
one (open circles in Fig. lc). As j3 increases, a plateau devel- 
ops in the center region from 25% up to 75% of cooperators, 
in which the fixation probability is roughly independent of k. 
For f3 — 0.05 its value is w 15%. For (3 — 0.1, the plateau 
widens, but more interestingly, it shifts position, so that now 
the fixation probability becomes w 2%. This behavior is re- 
lated to the location of k*, which in the finite system leads to 
the occurrence of an inflection point at k* in the fixation prob- 
ability curves. With increasing selection pressure the proba- 
bility distribution becomes sharply peaked around k*. Hence, 
the fixation probability of defectors rapidly approaches one, as 
k* < 1/2. In this case, the remnant of the interior fixed point 
leads to a fast increase of the time to fixation, which may be- 
come arbitrarily long (a detailed account of the fixation times 
will be published elsewhere 1 34]). This observation cannot 
be inferred from the weak selection analysis of Ref. fl9ll and 
contrasts sharply with the predictions from the replicator dy- 
namics equation: For the stochastic system, fixation in x = 
or x — 1 is certain, whereas the replicator dynamics predicts 
coexistence of cooperators and defectors. 

Throughout this work, we have followed the conventional 
approach of evolutionary game theory assuming that the pay- 
off matrices are constant and independent of j3. However, the 
intensity of selection and/or the payoff matrix may change 
during evolution, a feature which would lead to a coevolution- 
ary process that deserves further attention. One possibility is 
a situation in which there is a certain amplitude of variation in 
the payoff values during each interaction, for fixed (3, To the 
extent that these variations can be considered random, such an 
effect might contribute (in a mean-field treatment) to an over- 
all rescaling of the "selection temperature" leading to a new 
value j3' . Such new temperature would result from replac- 



ing the distribution of fitness values among each of the two 
types in the population by their average values. The theory 
presented here may be viewed as accounting for such a mean- 
field description, which will become more exact the narrower 
the payoff distributions. 

To sum up, the pairwise comparison process discussed here 
provides an unified framework in which one can study evo- 
lutionary dynamics in finite populations. We derived a sim- 
ple formula which determines the fixation probability for a 
given trait under frequency dependent selection, at all intensi- 
ties of selection and for any given number of initial individuals 
of that trait present in the population. In a finite population, 
the only stable evolutionary outcome corresponds to all indi- 
viduals sharing a common trait. The probability of fixation 
in either of these traits, however, is a non-trivial function of 
the number of individuals sharing that trait at start, depending 
sensitively on the intensity of selection, thereby configuring 
an interesting evolutionary scenario not accountable for pre- 
viously. We have shown that with increasing selection pres- 
sure the topological structure which is predicted by the deter- 
ministic replicator dynamics in infinite populations plays an 
increasing role. Depending on the nature of the game, we find 
that the transition between competing extrema in coordina- 
tion games exhibiting bistability gets sharper with increasing 
selection pressure. On the other hand, for games exhibiting a 
mixed equilibrium, the fingerprint of the stable mixed equilib- 
ria reflects itself in the rate of fixation into one of the absorb- 
ing states, which may increase prohibitively with increasing 
selection pressure. In other words, even when fixation is cer- 
tain, the time to reach it may become arbitrarily long. 
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